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ABSTRACT This paper suggests ways of structuring the content 

of feedback experimentally to test hypotheses of interest. Use of the 
feedback process in this manner suggests a variety of research and 
intervention tactics, many of them unexplored and some of -hem 
implying ethically questionable conduct, consideration of this 
cruestion also helps illuminate the potential conflict that exists 
between the ends of research and those of the client. It is JJJSff!®* 3 
that problems of this nature are endemic to projects which combine 
basic research and intervention-improvement goals. (Author) 
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USE AND IMPLICATIONS OF THE FEEDBACK 
PROCESS IN RESEARCH DESIGN 



Kenneth A. Sirotnik Mary L. Hruby . 

University of California ! l| D I E I A l Research Division 

Los Angeles Los Angeles 

During a long term intervention study (like SEC SI at |i|d|e|a)), the 
researcher is usually obligated to provide information to the subjects and/or 
system under study based upon some previous performance. This information 
or feedback can profitably be incorporated into the research design such that 
variations in the feedback along well-defined dimensions can be potentially related 
to variations in other riables of interest. The remainder of this paper will 
briefly consider: 1) the relationship of the feedback process to the independent 
and dependent variables in experimental design and, 2) some methodological 
implications resulting from experimental feedback intervention in on-going systems. 

It is usually the case that the variables involved in any given experimental 
design can be divided into two classes: U) maepenoent, or the variables fiseing 
experimentally manipulated and, (2) dependent, or the variables being affected by, 
and measured after said manipulation. Conceptualizing feedback as an independent 
or dependent variable is an important first step in incorporating feedback as a 
variable in research design. After taking this first step, we found that feedback 
could not be conceptualized as a dependent or independent variable. We found the 
exercise to be most important in that it served to elucidate some primary defini- 
tional problems with respect to the term feedback - problems which are easily 
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over -looked and only lead to semantic entanglements. Most importantly, we 
finally lost the struggle to conceptualize feedback as anything other than a process 
and not as a variable per se. Although we will attempt to deal with feedback as a 
" statistical" variable as rigorously as would appear to be meaningful, we are most 
concerned with emphasizing differentials in purpose of feedback as related to the 
research design rather than the nomenclature of independent versus dependent 
variable. We find it most profitable to discuss these differentials in purpose of 
feedback using illustrative research designs; furthermore, we find it most useful 
to draw upon our research activities at |i|d|e|a| since these experiences moti- 
vated our thinking of feedback as a research variable. 

For example, the SECSI study at |i|d|e| A| is attempting to promote a 
social structure among a set of elementary schools which will facilitate the process 
of dialogue, decision -making, and action among the members of any given school 
staff. It may often be the case during intervention strategies such as this to feed 
back results in different ways t( different e ^ ntal t its for the expressed 

purpose of looking for differential results in behavior at some later point in time. 
More specifically, suppose our experimental units are siaool faculties and we 
are interested in the "dialogue, decision-making, and action capacities" of each 
faculty. We construct a test to measure "DDA" and admmister it to all the school 
faculties. We then randomly divide the 18 schools into two groups of nine each. 

We report the results of the DDA mean scores to each of rhe schools in one of these 
groups (the "feedback" group) but not to those of the other group (the "no feedback" 
group). At the end of the semester we retest on DDA amd compare the two groups. 
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There are a host of experimental design problems associated with the 
content of this example. One thing should be clear, however, and that is that 
feedback has been related to an independent, experimentally manipulated variable. 

By feedback we mean the general process of feeding back information; if the 
independent variable in the above example were to be explicitly defined, it would 
not be feedback per se but the "amount of DDA test results fed back to school 
staff" -- a dichotomous variable taking on the values "all or none . 

We can also conceptualize the role of feedback in relation to the dependent 
variable in experimental design. In all experimental situations, the dependent 
variable is some psychological or behavioral construct (like achievement, attitude, 
etc.) which is measured (i.e. , inferred) from the experimental unit’s response to 
some standardized set of stimuli (e.g. , test items). In other words, the dependent 
variable is neither the test nor the response to the test but the inference we make 
with the response to the associated underlying psychological continuum. With 
these distinctions in mind, an analogy is offered between the use of feedback as a 
stimulus (or set of stimuli) in the same sense as are items on a test of achievement 

or attitude. 

Going back to the previous example, it is clear that, as stated, the dependent 
- variable is the construct DDA. But suppose we did not wish to compare the feedback - 
no feedback groups on the basis of DDA. Suppose, instead, we wished to compare 
them on some reaction to their DDA scores like, for example, "defensiveness" as 
measured by some Likert -hype attitude scale. In this situation, feedback (like items 
on a test) serves to elicit responses on the desired dependent variable continuum of 




interest. 
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It should be emphasized that in the first example, feedback was used 
differentially with respect to different experimental units. In the second situation, 
feedback was given consistently to all experimental units regardless of treatment. 
The purposes for feeding back information in each case are clearly different 
we wish to emphasize this difference in purpose for using the feedback process 
rather than the nomenclature of independent versus dependent variable. 

Implementing the process of feedback in research design is a simple decision 
that raises some complex problems. When the feedback process is an intrinsic part 
of an experimental design, the researcher must be aware of certain traditional 
methodological issues as well as those that emerge from the characteristics of a 
specific project. Traditional issues concern (1) Who is to give feedback? (2) In 
what form is it to be given? (3) When and in what setting is it to be given? (4) To 
whom is it to be given? Assuming feedback is to be used in conjunction with a 
treatment variable and to be followed by a period of measurement or observation of 
effects , the choice among alternatives for structuring feedback will influence the 
outcome. To be meaningful, the researcher must answer the above questions, 
implement his decisions in terms of the design of feedback, and apply it in standard 
fashion to all units. If this is done consistently, feedback can be related to 
observable outcomes, assuming other sources of variation have been controlled or 

eliminated. 

This faU, the SEC SI staff designed a feedback package based on the major 
problems identified by school staffs during spring, 1970. The content and format 
were standardized so that we could measure two outcomes: DDA and orientation 
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toward the League. A role-playing session was held to simulate the feedback 
situation in a school. The session and further staff discussion brought out 
several problems. Although the package was good from a research standpoint, 
its structure limited the amount of useable information it gave to the school staff. 

The SEC SI project combines both research and school improvement goals and some 
felt that research was being maximized at the expense of providing information 
that would facilitate school decision-making and problem-solving processes. In 
addition, use of the outside expert (i.e. , a SECS! staff member) as the communicator 
seemed inconsistent with our endeavors to lead school people to view and use one 
another as experts. The question of valid observation was also raised. Would we 
observe an atypical process, i. e. , response to the expert and not to the information? 
Using the principal as communicator would make the feedback situation more 

realistic since giving information to staff is a normal part of the principal's role. 

/ 

However, our decision to use the principal as communicator introduced his leader- 
ship and personal style as a source of variability (among others) whose effects on 
response to the "treatment" could only be controlled and measured ex post facto. 
That is , the cost for improving the utility and realism of the feedback process was 
in terms of reducing treatment standardization in the research design. Solving 
one problem raised another, a recurring situation in multi -goal projects such as 

SEC SI. 

Emergent issues are those which arise when project characteristics require 
consideration of the feasibility and/or desirability of various methods. The treat- 
ment - no treatment design is standard experimental practice. One can also vary 
the elements of a treatment across units to test for differential effects. A number 
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of alternatives are available for structuring feedback to test specific hypotheses. 

Basic to all experimental design is random assignment to treatment groups and 
no interaction between subjects in different groups. Considering the latter point 
first, how can this condition be met when a major project goal it> to promote 

I 

interaction among members? Many SEC SI interventions and activities are designed 
to foster intra- and inter -school communication. If different ’’feedback -related" 
treatments are applied, assessing effects is difficult given possible contamination 
due to subject interaction which we ourselves encourage! In addition, interaction 
alerts subjects to the fact that they are not receiving the same information. K 
people feel manipulated and come to distrust the researcher, it could shatter a 
once -good relationship, and even lead to their withdrawal from the project. This 
is the main reason that desirability as well as feasibility of methodological alternatives 

must be considered. 

A critical aspect of projects dealing with social experimentation and change 
is examination of the consequences of various methodological choices. Should the 
researcher manipulate feedback -related variables for research purposes when it is 
also intended to serve ameliorative purposes? What are the consequences of various 
strategies for maintenance of trust and rapport between researchers and clients? 

Do clients as "subjects" have any legitimate right to knowledge of the research 
design? 

Laboratory researchers are beginning to raise serious questions about the 
ethical implications of using various techniques for research with human subjects. 
Ethical problems become highly salient in on-going field studies which combine 
research and improvement objectives. If feedback is valuable information then 
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withholding it from some and giving it to others as a test of its effects is a 
questionable practice. Amount oi feedback on some variable as a treatment can 
be manipulated in various ways, all of which are legitimate on grounds that know- 
ledge is produced or enhanced. It can also be argued that the quality of knowledge 
is directly related to its value for the client. The point to be made here is that 
methodology is not neutral. The researcher, particularly in an on-going field 
situation must be aware not only of the ethical implications of his methodological 
choices, but also of the consequences of such choices for maintaining the research 
relationship and the commitment he has made to serve his clients. 

Kelman (1969:584) notes that "even under the most favorable conditions 
manipulation of the behavior of others is an ethically ambiguous act." The work 
of the behavioral scientist like that of the nuclear physicist requires that he "be 
concerned with the nature of the product(he is) creating and the social process to 
which (he is) contributing" (Kelman, 1969:583). A planned change project involving 
intensive researcher -client interaction introduces another dimension. Rc^ rding 
the manipulation of behavior, how knowledge is gathered becomes as important as 
its potential uses. While the researcher and his clients may have rapport and 
collaborate in the change effort, the values guiding behavior and the ends sought 
by each can never be wholly compatible and mutually reinforcing. The potential 
for conflict is increased when basic research and intervention into the client system 
are fused in one project (Smith, et. al . , 1960). Projects of this type have two 
primary goals: 

1. research on processes of behavioral and/or organizational change which 
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are of theoretical interest to the researcher in building the body of 
knowledge of his particular discipline. 

2. collection and application of such knowledge to alter or improve the 
functioning of the client system. 

The possibilities for conflict between these two goals have been examined using 

the feedback situation as an example. 

When the feedback process is part of the research design, its function is 

expanded beyond that of simply giving information to the client system. The 
researcher uses feedback content in a more or less controlled manner allowing him 
to observe how, why, or in what direction the system does or does not change in 
response to this input (Argyris and Taylor, 1960). Such information can have 
theoretical as well as diagnostic value: it broadens knowledge of social processes 
in general, and illuminates particular workings of the system(s) under study. 

Using schools as research sites, the type and content of feedback can, in 
theory, be structured so the researcher can compare and contrast its effects on the 
various units. A problem arises when one asks to what extent such uses of feedback 
violate the commitments of change agents to collaboration, effective change, and 
power equalization (Bennis, jetijil^, 1969: 580). Maximizing both theoretical and 
practical outcomes in one research design is possible but operationally unlikely. 
Methodologically, to design and assess the effects of treatments requires a degree 
of control often unreachable given differences among units (i.e. , 18 schools) and the 
particularistic nature of their needs and problems. 
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